Portable eye tracking device

ABSTRACT

A portable eye tracker device is disclosed which includes a frame, at least one optics holding member, a movement sensor, and a control unit. The frame may be a frame adapted for wearing by a user. The at least one optics holding member may include at least one illuminator configured to selectively illuminate at least a portion of at least one eye of the user, and at least one image sensor configured to capture image data representing images of at least a portion of at least one eye of the user. The movement sensor may be configured to detect movement of the frame. The control unit may be configured to control the at least one illuminator for the selective illumination of at least a portion of at least one eye of the user, receive the image data from the image sensors, and receive information from the movement sensor.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to Provisional U.S. Patent ApplicationNo. 61/873,154 filed Sep. 3, 2013, entitled “PORTABLE EYE TRACKINGDEVICE,” the entire disclosure of which is hereby incorporated byreference, for all purposes, as if fully set forth herein.

This application is also related to U.S. patent application Ser. No.______ filed concurrently herewith, entitled “PORTABLE EYE TRACKINGDEVICE,” (Attorney Docket No. 89056-908793), U.S. patent applicationSer. No. ______ filed concurrently herewith, entitled “PORTABLE EYETRACKING DEVICE,” (Attorney Docket No. 89056-909170), and U.S. patentapplication Ser. No. ______ filed concurrently herewith, entitled“PORTABLE EYE TRACKING DEVICE,” (Attorney Docket No. 89056-909171), theentire disclosures of which are hereby incorporated by reference, forall purposes, as if fully set forth herein.

BACKGROUND OF THE INVENTION

Methods for determining a point at which a user is looking are known inthe art. The technology is often referred to as eye tracking or gazedetection technology. Eye tracking systems are typically found in twomajor forms. In one form, a sensor or other eye tracking device islocated remote from a user's eye such as in or on a computer, display orsimilar device.

One known method of eye tracking includes the use of infrared light andan image sensor. The infrared light is directed towards the pupil of auser and the reflection of the light is captured by an image sensor.Through analysis of the reflection point, the direction of the user'sgaze may be calculated. One such system is described in U.S. Pat. No.7,572,008 assigned to Tobii Technology AB, Sweden (the “'008 patent”).The entire disclosure of the '008 patent is hereby incorporated byreference, for all purposes, as if fully set forth herein.

Portable or wearable eye tracking devices have also been previouslydescribed and are available for commercial purchase. One such eyetracking system is described in U.S. Patent Application PublicationNumber 2011/0279666 assigned to Tobii Technology AB, Sweden (the “'666application”). The entire disclosure of the '666 application is herebyincorporated by reference, for all purposes, as if fully set forthherein. The '666 application describes a wearable eye tracking devicethat requires an external infrared light source to be placed in a sceneas a reference point, to assist in determining the direction of a user'sgaze.

Existing portable eye tracking systems may suffer severe performancedegradation when the equipment moves relative to the wearer's head. Forexample, glasses may slip relative to a wearer's nose; in addition, awearer may manually adjust glasses as they are worn. For designsrequiring calibration, such movement of the glasses relative to thewearer's head may negate the calibration and significantly degrade theaccuracy of the readings. As another example, a single-camera portableeye tracking system may render substantially degraded reading in certainconditions including when the wearer is in the presence of strong lightsources; is exposed to direct sunlight; or when the single camera's viewis obstructed such as by an eyelash. Furthermore, such single-camerasystems may be unable to detect gaze directions at the extremities of auser's field of view.

BRIEF SUMMARY OF THE INVENTION

In one embodiment, a portable eye tracker device is provided. Theportable eye tracer device may include a frame, at least one opticsholding member, a movement sensor, and a control unit. The frame may bea frame adapted for wearing by a user. The at least one optics holdingmember may include at least one illuminator configured to selectivelyilluminate at least a portion of at least one eye of the user, and atleast one image sensor configured to capture image data representingimages of at least a portion of at least one eye of the user. Themovement sensor may be configured to detect movement of the frame. Thecontrol unit may be configured to control the at least one illuminatorfor the selective illumination of at least a portion of at least one eyeof the user, receive the image data from the image sensors, and receiveinformation from the movement sensor.

In another embodiment, a method of determining a gaze direction for auser is provided. The method may include activating at least oneilluminator on a frame worn by a user to selectively illuminate at leasta portion of at least one of the user's eyes. The method may alsoinclude receiving image data representing images at least a portion ofat least one of the user's eyes from at least one image sensor on theframe. The method may further include receiving information from amovement sensor configured to detect movement of the frame. The methodmay additionally include determining a gaze target area for the userbased at least in part on the image data and information from themovement sensor.

In another embodiment, a non-transitory machine readable medium havinginstructions thereon for determining a gaze direction for a user isprovided. The instructions may be executable by a processor foractivating at least one illuminator on a frame worn by a user toselectively illuminate at least a portion of at least one of the user'seyes. The instructions may also be executable for receiving image datarepresenting images at least a portion of at least one of the user'seyes from at least one image sensor on the frame. The instructions mayfurther be executable for receiving information from a movement sensorconfigured to detect movement of the frame. The instructions mayadditionally be executable for determining a gaze target area for theuser based at least in part on the image data and information from themovement sensor.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention is described in conjunction with the appendedfigures:

FIG. 1A shows an eye tracking device according to some embodiments ofthe present invention;

FIG. 1B shows an alternative eye tracking device according to otherembodiments of the present invention;

FIG. 2A shows a rear view of an eye tracking device according to someembodiments of the present invention;

FIG. 2B shows a rear view of an eye tracking device according to otherembodiments of the present invention;

FIG. 3 shows a first view of an optics holding member according to someembodiments of the present invention;

FIG. 4 shows a second view of an optics holding member according to someembodiments of the present invention;

FIG. 5 shows one possible method employed by various embodiments of theinvention; and

FIG. 6 is a block diagram of an exemplary computer system capable ofbeing used in at least some portion of the apparatuses or systems of thepresent invention, or implementing at least some portion of the methodsof the present invention.

In the appended figures, similar components and/or features may have thesame numerical reference label. Further, various components of the sametype may be distinguished by following the reference label by a letterthat distinguishes among the similar components and/or features. If onlythe first numerical reference label is used in the specification, thedescription is applicable to any one of the similar components and/orfeatures having the same first numerical reference label irrespective ofthe letter suffix.

DETAILED DESCRIPTION OF THE INVENTION

The ensuing description provides exemplary embodiments only, and is notintended to limit the scope, applicability or configuration of thedisclosure. Rather, the ensuing description of the exemplary embodimentswill provide those skilled in the art with an enabling description forimplementing one or more exemplary embodiments. It shall be understoodthat various changes may be made in the function and arrangement ofelements without departing from the spirit and scope of the invention asset forth in the appended claims. For example, any detail of oneembodiment discussed herein may or may not be present in all possiblevariations of that embodiment, and may or may not be present in allpossible variations of other embodiments discussed herein.

Specific details are given in the following description to provide athorough understanding of the embodiments. However, it will beunderstood by one of ordinary skill in the art that the embodiments maybe practiced without these specific details. For example, circuits,systems, networks, processes, and other elements in the invention may beshown as components in block diagram form in order not to obscure theembodiments in unnecessary detail. In other instances, well-knowncircuits, processes, algorithms, structures, and techniques may be shownwithout unnecessary detail in order to avoid obscuring the embodiments.

Also, it is noted that individual embodiments may be described as aprocess which is depicted as a flowchart, a flow diagram, a data flowdiagram, a structure diagram, or a block diagram. Although a flowchartmay describe the operations as a sequential process, many of theoperations can be performed in parallel or concurrently. In addition,the order of the operations may be re-arranged. A process may beterminated when its operations are completed, but could have additionalsteps not discussed or included in a figure. Furthermore, not alloperations in any particularly described process may occur in allembodiments. A process may correspond to a method, a function, aprocedure, a subroutine, a subprogram, etc. When a process correspondsto a function, its termination corresponds to a return of the functionto the calling function or the main function.

The term “machine-readable medium” includes, but is not limited toportable or fixed storage devices, optical storage devices, wirelesschannels and various other mediums capable of storing, containing orcarrying instruction(s) and/or data. A code segment ormachine-executable instructions may represent a procedure, a function, asubprogram, a program, a routine, a subroutine, a module, a softwarepackage, a class, or any combination of instructions, data structures,or program statements. A code segment may be coupled to another codesegment or a hardware circuit by passing and/or receiving information,data, arguments, parameters, or memory contents. Information, arguments,parameters, data, etc. may be passed, forwarded, or transmitted via anysuitable means including memory sharing, message passing, token passing,network transmission, etc.

Furthermore, embodiments of the invention may be implemented, at leastin part, either manually or automatically. Manual or automaticimplementations may be executed, or at least assisted, through the useof machines, hardware, software, firmware, middleware, microcode,hardware description languages, or any combination thereof. Whenimplemented in software, firmware, middleware or microcode, the programcode or code segments to perform the necessary tasks may be stored in amachine readable medium. A processor(s) may perform the necessary tasks.

Referring now to FIG. 1A a portable eye tracking device 10 according toone embodiment of the invention is shown. Eye tracking device 10includes a frame 12 having a glasses lens (or pair of lenses) 14 coupledwith it in such a manner so as to resemble a traditional pair of eyeglasses or sunglasses. Frame 12 may include a center piece 15 to whichglasses lens 14 is attached and from which two arms 17 (i.e., temples)extend. Arms 17 may fit above the ear of a user in the manner of atraditional pair of eye glasses or sun glasses.

In some embodiments, a nose piece 16 may be provided for comfort and toassist in fitting of device 10 to a user's nose. A scene camera 18(which may capture still and/or video images and/or other image data)may be disposed in the middle of glass lens 14, or between separateglass lenses, above the nose piece 16. A microphone 20 may also beplaced adjacent or near to scene camera 18.

A control unit 22 may be located within one or both arms 17, either inthe front, middle, and/or end thereof. Any processor function describedherein may be executed at control unit 22, and/or an external processorin communication with control unit 22. Control unit 22 may includeelements for performing computational tasks, such as a printed circuitboard (PCB) and other electronics, as will be described in furtherdetail herein. Control unit 22 may also contain a communications port orcomponent designed to communicate with an external computing device.This communications port or device may support any one or more form ofknown communication. For example, a communications port may include aUniversal Serial Bus (USB) port, Firewire port, High-DefinitionMultimedia Interface (HDMI) port, Ethernet port or the like. Acommunications device may include a Wi-Fi transceiver, Bluetoothtransceiver, or any other near-field or longer-range communicationdevice. In other embodiments, the communications port or device may alsobe of a proprietary type especially designed for use in a portable eyetracking device. The communication port or device may for exampleinclude a low-powered wireless communications means.

FIG. 1B, shows an alternative eye tracking device 10A with certaincomponents in different positions. In this embodiment, control unit 22may be located on the side of arm 17, and include an HDMI interface 19.

FIG. 2A shows eye tracking device 10 according to certain embodiments ofthe invention as viewed from behind. For illustration purposes a portionof center piece 12 has been removed from the figure to show theplacement of a circuit board 50 which may contain electronic componentssuch as flex connectors 52 and microphone 20. Circuit board 50 may alsocontain processing architecture such as a digital signal processor,field programmable gate array and/or another processor system on a chip.Circuit board 50 may be attached to center piece 12 by way of aconventional fastening means, such as screws 54, adhesive, and/or othermeans. Center piece 12 may include one or more parts which fit overcircuit board 50, such that circuit board 50 is disposed within centerpiece 12, and not visible during normal use.

Device 10 may also include two optics holding members 70. In otherembodiment only one contiguous optics holding member may be providedwhich provides the functionality of the two optics holding members 70shown in FIG. 2A. Each optics holding member 70 may include one or moreimage sensors and one or more illuminators as will be further explainedbelow. Thus, an optics holding member 70 may include components thatilluminate a user's eye, or some sub-portion thereof (for example, theiris), and capture images comprising reflection points of theillumination on the user's eye, or some sub-portion thereof. FIG. 2B,shows the alternative eye tracking device 10A of FIG. 1B.

FIGS. 3 & 4 show optics holding members 70 according to embodiments ofthe invention in further detail. Each optics holding member 70 mayinclude two primary components, a cover 72 and a flexible member 74.Cover 72 may cover flexible member 74 when properly coupled together asshown by optics holding member 70B. Optics holding member 70A is shownwith flexible member 74 and cover 72 parted in order to show theinteraction between the two components. Flexible member 74 and cover 72may be configured such that the flexible member 74 snaps into the cover72. As another example, the flexible member 74 and cover 72 may beconnected to one another using adhesive, screws and/or otherconventional fastening means.

Flexible member 74 may include a flex circuit and have mounted on it oneor more spaced illuminators 76 and/or one or more image sensors 78. Anynumber of illuminators 76 (e.g., 1, 2, 3, 4, 5, 6, etc.) and imagesensors (e.g., 1, 2, 3, 4, 5, 6, etc.) may be employed in each opticsholding member 70. Illuminator 76 may include infrared illuminatorscapable of emitting infrared light as would be readily understood by aperson of skill in the art, but other illuminators may also be used,including illuminators emitting ultraviolet and/or visible light. Imagesensor 78 may be sensitive to the light emitted by illuminator 76 aswell as visible light and other light that may be present in the area ofuse. For example, when device 10 is used outside, ambient light in thevisible and nonvisible spectrums (e.g., visible and ultraviolet lightfrom the sun) may be present and image sensor 78 may be configured todetect and distinguish (e.g., filter out or otherwise compensate for)different wavelengths of ambient light. Flexible member 74 may include apower source (e.g. batteries, solar cell, etc.) to provide power toilluminator 76 and image sensor 78 in a manner that would be wellunderstood by a person of normal skill in the art. An upper end of theflexible member 74 may include a contact element 80 that connects tocircuit board 50 mounted on center piece 12. The use of flexiblecircuits and electronics is well known, and a person skilled in the artwould appreciate the way they can be used in the present invention.

In some embodiments of the invention, cover 72 may be formed from anoptically transparent material such that light from illuminators 76 maypass through cover 72 substantially unhindered. In some embodiments,cover 72 may be shaped to transmit the light emitted by illuminators 76to the appropriate location. Parts or areas of cover 72 may for examplebe shaped as a lens that diverges light from one or more of illuminators76 (or even ambient lighting sources) to illuminate a larger areaincluding the user's eyes. Parts or areas of cover 72 may also be shapedto converge or focus light on particular areas or locations. Cover 72may be molded out of a single piece of material wherein areas of thematerial are shaped like lenses or otherwise shaped to transmit light asdescribed above. In other embodiments the cover may include multipleparts that are affixed together by glue, welding, screws and/or otherconventional fastening means, where some of the parts transmit light andsome do not, or where different parts transmit light in different ways.In some embodiments, flexible member 74 and/or cover 72 may be doublemolded and include filters which prevent transmission of light moredirectly from illuminators 76 to image sensors 78 through cover 72.Optical windows may be provided by cover 72 or elsewise at each imagesensor 78.

Eye tracking device 10 may emit light from illuminators 76 thatilluminate at least part of at least one of the user's eyes. One or moreimage sensors 78 may then capture an image comprising at least theportion of the eye as illuminated by illuminators 76. This capturedimage may be transmitted via the flexible member 74 to processingdevices (e.g., control unit 22 or other processor, perhaps in a deviceseparate from frame 12), where the direction of the gaze of the user maybe determined by analyzing the image data.

To determine the gaze direction of the user, a cornea position of one orboth eyes of the user may be determined. The cornea position of theuser's eye(s) may be analyzed by detecting glints or reflections of thelight that is emitted from illuminators 76 onto the eye(s) of the user.In order to obtain a high quality image showing a glint on the user'seye, various combinations of multiple illuminators 76 may be used toemit light. For example, one illuminator 76 may illuminate usinginfrared light, while another illuminates using another wavelength oflight. The image sensors 78 may then capture images when the eye isilluminated with ambient light only, when the eye is illuminated by eachilluminator 76 individually, or when the eye is illuminated by two (ormore) illuminators 76 at the same time. In such embodiments, imagesensors 78 may prepare differential images of the user's eye.

Detection of glints may be done with image differentiation techniques(i.e., comparing a first image to a second image to detect a change)and/or standard image analysis algorithms. For example, by toggling anilluminator 76 on and off and capturing an image of the user's eye ineach state, the resulting glint may be detected via comparison of theimages. However, in some embodiments, glints may be detected byanalyzing an one or more images to identify areas of intensitycorresponding to a glint. One method of glint detection is outlined inthe article “Improved Video-Based Eye-Gaze Detection Method,” byYoshinobu Ebisawa, published on 4 Aug. 1998, which is herebyincorporated by reference, for all purposes, as if fully set forthherein. Further methods of glint detection are discussed in U.S. Pat.No. 6,577,329 titled “Method and system for relevance feedback throughgaze tracking and ticker interfaces” and U.S. Pat. No. 8,292,433 titled“Method and apparatus for communication between humans and devices.” Theentire disclosures of the aforementioned patents are hereby incorporatedby reference, for all purposes, as if fully set forth herein. Personsskilled in the art will be aware of a number of ways of determining agaze direction from light reflecting from a user's eye, and the presentinvention is not limited to the examples recited above.

In some embodiments of the invention, a given number of infraredilluminators 76 may be used to illuminate each of a user's eyes. Asdiscussed, other numbers and/or types of illuminators 76 may be used inother embodiments. In such embodiments one or more glints correspondingto each illuminator 76 would be expected. As there may also be otherglints resulting from ambient lighting, such as a spotlight or sunlight,etc., different techniques may be used to identify which glintscorrespond to the illuminators 76, and which ones do not. In someembodiments, an image taken of the user's eye without the illuminators76 turned on may be compared to one taken with the illuminators 76turned on to filter out glints caused by ambient light. However, inother embodiments, the size, shape, expected intensity, and expectedpositions of glints may be used to determine which glints correspond towhich illuminators.

In other embodiments, a wavelength filter may be used in conjunctionwith image sensors 78 to filter out wavelengths of light that do notcorrespond to the wavelengths emitted by illuminators 76. For example,where illuminators 76 emit infrared light, a filter that passes onlyinfrared light through to image sensor 78 may be used. In this way,image sensors 78 may only detect glints resulting from light emitted bythe illuminators 76. Conversely, filters may be employed that filterinfrared light emitted by illuminators 76 while passing ambient light.Such filters may work well with respect to various artificial sources ofambient light. In some embodiments, lenses 14 may also be configured toblock ambient infrared or some other wavelength of light However, in thecase of direct sunlight, which includes a spectrum of light comprisingboth infrared and ultraviolet, filters may not be able to adequatelyblock all of the ambient light and pass only the light emitted byilluminators 76. As a result, glints resulting from ambient light maynot be distinguishable from glints resulting from light emitted byilluminators 76 when the illuminator is lit.

Ambient light conditions for a user of wearable eye-tracker device 10may change drastically over time. For example, if the user is facing thesun, his or her eyes may be subject to substantial illumination bysunlight, whereas if the user is facing away from the sun, his/her eyesmay be significantly less illuminated. Similarly, if the user is in anindoor environment, the illumination may vary significantly based on theproximity of various light sources. For example if the user is standingdirectly below a ceiling light, his/her face and eyes may besubstantially more illuminated than if he/she were standing adjacent tothe ceiling light. Furthermore, in some embodiments, depending on theplacement and types of light sources, the levels of ambient light may bedifferent between the two eyes being tracked. For example, the usermight be positioned such that his/her left side is in direct sunlightwhile his/her right side is in a shadow.

As discussed, it may be possible to compensate for some changes inambient light levels by using the built in illuminators 76. It mayfurther be possible to counter the effect of ambient light variations byusing light outside the frequency range of light available in theambient lighting setting. For example in an indoor setting, ultravioletilluminators 76 may be used to illuminate the user's eyes withultraviolet light that is not present or is present at lower levels inthe indoor lighting. Similarly, infrared illuminators 76 may be used onsome embodiments, given that infrared light is typically present at lowlevels in indoor settings. In some embodiments, illuminators 76 capableof emitting light over a range of wavelengths may be used. In suchembodiments, the device 10 may be programmed to dynamically analyze thespectrum of ambient light, and select a wavelength to be emitted byilluminators 76 that is higher or lower than the detected spectrum orotherwise confined to a certain part of the detected spectrum, in whichcase sensor 78 may also be dynamically adjusted by the device 10.Alternatively, more than one sensor 78 may be provided of differenttypes attuned to different wavelengths.

In another embodiment, a wavelength of light may be employed which has ahigh level of absorption in water, such as 940 nm. This concept isdiscussed in European Patent Application No. 12192370.0, the entiredisclosure of which is hereby incorporated by reference, for allpurposes, as if fully set forth herein. At this wavelength the darkpupil effect of the eye may be at its maximum as a majority of lightentering the eye will be absorbed. Additionally, ambient light levels inthe atmosphere at this wavelength are relatively low. Further, ambientlight may be addressed by providing glasses lens 14 in a suitablematerial or configuration to block light at the wavelength visible tosensor 78. Functionally this improves the signal quality as light isblocked from passing through the glasses lens 14.

In some embodiments, differential lighting applications may be used. Forexample, image sensor(s) 78 may capture images of the user's eye 60times per second and illuminators 76 may be configured to change state(on/off) 60 times per second out of phase with exposure of sensor(s) 78.In such a scenario, every other frame of image data represents theuser's eye as illuminated by illuminators 76, with the alternate framesrepresenting the user's eye as illuminated by ambient light. Whenanalyzing the image data, image processing may be employed to determinethe difference between two adjacent frames and thus differentiate glintscaused by reflection from illuminators 76 from those caused byreflection from ambient light. The intensity or exposure time ofilluminators 76 may be adjusted dynamically in some embodiments, forexample based on the level of ambient light around the wearer. Forexample, a higher level of illumination, or longer exposure time, may beapplied when more ambient light is present, or a lesser level ofillumination, or shorter exposure time, may be applied if glints fromambient light are advantageously used by the sensors. Furthermore,illuminators 76 may be controlled in groups to allow image sensors 78and image processing algorithms to detect and compensate for noise fromambient light.

Ambient light may be used to detect contrasts in captured image data todetermine the position of the user's pupil and/or iris. This informationmay be used together with information based on glints associated withilluminators 76 to determine the direction in which the user is lookingDifferent settings for pulse length and intensity of the illuminationemitted from illuminators 76 and the exposure time of image sensors 78may be used to compensate for brightness of ambient light, and may inparticular improve performance in dark or bright ambient lightconditions.

For example, when there is a high level of ambient light, illuminators76 may be disabled and the position of the user's pupil may betriangulated based on glints or other image data detected as a result ofthe ambient light. Thus, image data may be analyzed to determine thelocation of the cornea using only ambient light. In some cases,illuminators 76 may be used to assist the contrast detection, with shortflashes of illumination coordinated with image capture by image sensors78 may be sufficient to detect the position of the user's pupils. Theuse of short intensive illumination pulses with shorter exposure timesmay help avoid effects such as over-exposure, motion blur and rollingshutter effects. For example, in a shopping research scenario, a mobileeye tracker user may move his/her head around quickly while scanning thestore shelves for a desired product. Similarly, when there is too littleambient light in the environment, the illuminators 76 may be used tohelp produce light to make it possible to pick up the contrasts of theeye. Illuminators 76 may thus be configured to emit lower intensitylonger pulses with longer exposure times, and/or constantly emitting lowintensity light. In one embodiment of the present invention, allilluminators 76 may be activated concurrently such that multiplereflections on the cornea of emitted light may be captured by sensors78. Device 10 may then use measured pupil position to determine one ormore of gaze direction, orientation data, cornea position, and pupilsize.

In some embodiments, a calibration of at least one of the plurality ofilluminators 76, at least one of the plurality of image sensors 78, oran algorithm of the control unit 22 may be conducted in controlledlighting conditions to determine the location of the pupil and orcornea. In yet other embodiments, the location of the iris may betracked based on ambient light alone; for example the lines of the irismay be identified to determine the orientation of the eye. Various modesthat rely on ambient light only may be activated based on a number ofconditions. For example, such a mode may be activated when battery poweris low so as to conserve power by disabling illuminators 76.Furthermore, the mode may be activated when ambient lighting reaches alevel to where the accuracy of glint-based tracking is at leastcomparable to the accuracy of another available mode. Persons skilled inthe art will appreciate that a number of different configurations may beused to obtain image data, and that the invention is not limited to theexamples recited above. Additional calibration methods will be furtherdiscussed herein.

Accuracy of portable eye tracker device 10 may be maximized by ensuringthat the glints are of optimal size. Glints that are too small may bedifficult to detect and therefore reduce accuracy. On the other hand,glints that are too large may be difficult to accurately place and maytherefore similarly reduce performance by interfering with pupildetection or otherwise. The size of the glints may be affected by theintensity of light emitted by illuminators 76. The size of the glintsmay further be affected by the sensitivity settings, aperture and/orexposure time for image sensors 78. The setting(s) for image sensors 78may be adjusted to compensate for ambient light, and the intensity ofilluminators 76 may in turn be adjusted to obtain the optimal glintsize. Furthermore, the intensity of illuminators 76 and the settings forimage sensors 78 may be balanced to obtain the optimal contrast level ofthe glints such that they can be easily detected. A person skilled inthe art will appreciate that the size, contrast and othercharacteristics of glints may be optimized in a number of ways, and thatthe invention is not limited to the examples recited above.

The light conditions for each eye can be determined by analyzing imagescaptured by image sensors 78 or using external light sensors, or acombination of both. The result of this analysis may be used to controlthe eye tracker settings for each eye in real-time. In some embodimentsof the invention, the relevant light levels may be determined by lightsensors embedded in portable eye tracker device 10. In otherembodiments, the light levels may be determined using one or more scenecameras 18. By individually controlling the illuminators in real-timebased on the current light conditions for each eye, overall performanceof the eye tracker device 10 may be improved compared to situation inwhich the settings are based on only one eye or an average of both eyes.

After the glints corresponding to illuminators 76 have been identifiedin the image data captured by image sensor 78, the location of suchglints relative to the user's eye is determined using known imageprocessing techniques. Once the glint locations are determined, they maybe analyzed to determine the position of the user's cornea. In someembodiments, the locations of the glints may be mapped on to athree-dimensional model of the human eye. For example, the glints may bemapped onto the cornea. In some embodiments the cornea may be assumed tobe a perfect sphere; the locations of the glints may be used todetermine the location of the cornea relative to the pupil. Thislocation of the cornea relative to the pupil may in turn be used todetermine the direction of the gaze and thereby the optical axis of theeye. Various means of determining a gaze direction based on glints areknown in the art and a person skilled in the art would appreciate thatthe present invention is not limited to the examples recited above.

In some embodiments of the invention, more than one image sensor 78 isprovided for capturing images of each eye. Where two image sensors 78are used for each eye, this may be referred to as “stereo mode.” Bycapturing images of an eye from multiple points of view, additionalinformation may be determined by processor such as control unit 22 orother processor, such as the distance of the eye from each image sensor78. Further, by operating more than one image sensor 78 there is a levelof redundancy in the system whereby it may still function even if one ormore image sensor 78 cease to function.

Any number of image sensors and/or configuration thereof may be combinedwith multiple illuminators 76 to operate in various configurations tooptimize the reliability of the system. For example, device 10 may beconfigured to try various configurations of illuminators 76 (e.g., cyclethrough various illumination patterns/sequences) to determine whichconfiguration of illuminators 76 creates the best image at each imagesensor 78. Illuminators 76 may then be configured to change state(on/off) at a frequency such that each image sensor 78 takes a picturein optimal illumination. For example, if there are three image sensors78 for each eye, and each image sensor 78 captures 30 images per second,illuminators 76 may be configured to change state 90 times per second sothat each image sensor 78 can capture an image in a dedicatedillumination setting. This may further be used to provide an increasedframe rate for the gaze data. Further, using more than one image sensor78 for an eye reduces the necessity of calibration between the device 10and the user, i.e., it may be possible to operate the device 10 withoutuser calibration in some instances due to the extra data gathered byhaving more than one image sensor 78.

More than one image sensor 78 may also allow for compensation ofphysical movement of the device 10. For example, device 10, whenconfigured as a pair of glasses, may slide down the user's nose, and theuser may in turn push it back up. Device 10 may also move relative tothe user's head after rapid movement of the user's head or for any otherreason. This type of movement of device 10 relative to the user's headmay reduce or eliminate any accuracy gained from prior calibration.Using multiple image sensors 78 may improve accuracy without the needfor recalibration. For example when two image sensors 78 are used foreach eye, each glint is detected from two perspectives and it ispossible to estimate the location of the cornea with greater accuracyeven in cases where device 10 has moved relative to the user's headafter any initial calibration.

In some embodiments of the present invention, device 10 further includesone or more movement sensor or positioning device 25. Movement sensor orposition device 25 may include one or more of a gyroscope; anaccelerometer; a compass, a GPS or other satellite receiver; GLONASScompass; or any other location, positioning, or direction sensor.Movement sensor or positioning device 25 may enable tracking of theposition and/or orientation of device 10 itself and in turn the locationor position of the user's head. This may allow the device 10 to accountfor head movement and adjust gaze direction data based on theinformation about head movement to provide a better estimate of thedirection of the gaze. A gaze direction determined based on theorientation of the user's eyes may be relative to the orientation of theuser's head. Information about the gaze direction based on the user'seyes may therefore be augmented with information about the orientationof the user's head. For brief moments, this augmentation may beperformed using an accelerometer or a gyroscope. However, theinaccuracies of these devices may lead to drift if they are used todetermine the orientation over time. In some embodiments informationabout gaze direction based on the user's eyes and/or information aboutthe orientation of the user's head may be further augmented usinginformation from a compass which provides orientation informationrelative to absolute points of reference. Additionally information aboutthe change in orientation of the device may be helpful when conductingfixation filtering of gaze data. Assuming that objects in thesurrounding stay relatively stationary it may be much easier todetermine that the user wearing the device fixates at a stationaryobject while moving their head if device orientation data is available.

In other embodiments, movement sensor or positioning device 25 may beembedded in an apparatus connected with device 10 or any portion ofdevice 10. For example, movement sensor or positioning device 25 may beconnected with device 10 or any portion of device 10 wirelessly or witha cable. In some embodiments, such movement sensor or positioning device25 may be carried in a backpack worn by the user, or otherwise carriedby the user. When movement sensor or positioning device 25 is embeddedin the device 10 it may be able to provide more accurate informationabout the location of the user's gaze. A number of different systems canprovide or account for the required latency and accuracy. In particular,gyroscopes and compasses may provide more accurate information ifembedded in the device 10 itself.

Furthermore, information from movement sensor or positioning device 25may be used to stabilize image data from scene camera(s) 18. Forexample, scene camera 18 may capture image data approximating the viewseen by the user. Information from movement sensor or positioning device25 may be used to stabilize this image data. Information about theuser's gaze may further be used to crop or otherwise adjust this imagedata to more accurately represent the gaze direction of the user. Imagestabilization or compensation may include line-shifting of the videodata. There are a number of well-known image stabilization methods thatinvolve the use of a gyroscope or accelerometer, and a person skilled inthe art will appreciate how these methods may be combined withembodiments of the present invention. Furthermore, time basedcompensation may be applied to the image data to account for imagecapture delay inherent in scene camera 18. In particular, imagescaptured by video cameras, rolling shutter cameras as well as CCD typecameras may not show the correct view for when a gaze point was obtaineddue to image capture delay. Time compensation may differ depending onthe type of scene camera 18 and/or where in the image the gaze point ispositioned.

Image processing and stabilization may in some embodiments be performedin real-time on device 10, or on an external device, as the data iscaptured. In other embodiments, the image data may merely be stored ondevice 10 or transferred to an external device (not shown), and theimage stabilization or other processing may be performed later based onthe captured data. The approach taken may depend on the processing poweravailable in device 10, as well as the energy available from on-board orotherwise connected power source(s). In particular, certain types ofprocessing may require large amounts of computing power that in turnconsume a lot of battery capacity. In some embodiments device 10 may beconfigurable to either optimize battery capacity or to optimize forreal-time processing.

Furthermore, elements in the captured image or video from scene camera18 may be analyzed to determine the orientation of a user's head and thespeed of movement of the user's head. By analyzing the relative positionof elements in successive images or video, adjustments to thecalculation of a gaze direction may be made to compensate for movement.In further embodiments, image sensors 78 and/or the scene camera 18 mayutilize a rolling shutter to further improve the accuracy of theaccuracy of determining the orientation of the users head. Bydetermining the orientation and movement of a user's head in combinationwith the readout information of rows of image sensors 78 and scenecamera 18, the determined gaze direction may be overlaid on an imagecaptured by scene camera 18 in such a way that the gaze direction may becorrected to reflect the actual scene present in the user's field ofview at the time the gaze direction was calculated.

For example, device 10 may calibrate captured data from scene camera 18with gaze data derived from the sensors 78 so as to more correctlyreflect where and when a user was looking at a particular time inrelation to captured data from scene camera 18. In embodiments of theinvention, this approach may be adopted where image data captured byscene camera 18 has been distorted due to, for example, rolling shutterdistortion or vibrations. Further embodiments include consideringinstantaneous movement data of the device and calibrating accordingly byutilizing a motion sensor and/or other sensor.

Video from scene camera 18 may be analyzed to identify objects in theuser's field of view. By identifying the objects and the distance of theobjects from the user, more accurate information regarding the directionand target of the gaze may be determined. Computer vision algorithms maybe used to detect objects from the image data. In some embodiments,multiple scene cameras 18 may be used to provide stereo vision and moreaccurate computer vision algorithms. As with image stabilization andother processing, object identification may be done in real-time or aspost processing depending on the computing power and power capacityavailable on device 10. Alternatively or additionally, scene camera 18may be a depth camera measuring the distance to objects within the fieldof view of the person wearing the device 10. The depth camera may alsodetermine the intensity level of the objects, thus also provide a greyscale image.

Furthermore, image analysis may be used to determine the location of theuser based on identification of objects detected within the user's fieldof view. In other words, an object within the user's field of view,e.g., a landmark or object known to be associated with a given place ortype of place may reveal the user's location. For example an object orother indicia, such as signage, types of products, pricing labelsproducts, etc. may indicate that the user is present within a particularretail store or at least a type of retail store. As another example,scene camera 18 or another scanning device connected to device 10 may beconfigured to scan bar codes that appear within the user's field ofview, which may reveal that the user is present within a retail store orother known location or type of location. Such location information maybe compounded with information from location sensors and movementsensors to determine a path taken by a user to get to his/her presentlocation and/or to navigate around that location. Where a user is in thevicinity of a television or display, for example displaying a computergame, the game may be able to process eye tracking input provided bydevice 10; image analysis may be used to determine the direction of theuser's gaze relative to the television or display.

Furthermore, image analysis of images from scene camera 18 may beutilized for simultaneous localization and mapping (SLAM). SLAM is theprocess of building a map of a location while simultaneously mapping theposition of a device within that map. SLAM is frequently used by robotsand similar appliances and may include an image sensor for capturingimages of the robot's environment for mapping. A device according to anembodiment of the present invention may also be used with SLAM.

Where the motion sensors are used to detect motion, there may beconsiderable drift within the data when they are relied on for longerperiods of time. The information from these motion sensors may thereforebe corrected by location information determined based on objectsdetected from scene camera(s) 18. Objects detected using scene camera(s)18 may include characteristic structures or objects as well as barcodes. In other embodiments, sound information detected via microphone20 may be used to determine a location. For example ultrasound emittersmay be placed at various points throughout a particular location, suchas a retail store, and microphone 20 may be used to determine theclosest emitter. Alternatively the ultrasound source may be mounted ondevice 10 and the microphone used to determine the distance to theclosest object in the direction of the ultrasound source. In addition orin the alternative, microphone 20 may be used to detect other ambientsounds and such information may be used, at least in part, to determinea location or type of location in which the user is present. In someembodiments, an RFID tag reader may be included in device 10 so thatRFID tags can be used to determine the user's location. As an additionalexample, Wi-Fi signals and/or other communication signals may bereceived and triangulated by appropriate transceivers and logic onboardthe device 10 to determine the user's location. A person skilled in theart will appreciate that the invention is not limited to the examplesrecited above and that a number of location identification means may beused to aggregate or determine location information about the wearer.

In some embodiments, one or more additional devices may be embedded inor coupled with device 10. For example, scene camera 18 may be used torecord images in an area in which the user might move and/or look. Agyroscope may be used for a compass-like feature to identify whichdirection device 10 is pointing and thus in which direction the user islooking. Image sensors 78 in the device may then identify the angle anddistance of the user's gaze point based on the direction of where thehead is pointing. Information from these multiple sensors in combinationmay be used to compute a vector representing the wearer's gaze. Thisvector may be transformed and visualized in the view of scene camera 18.The vector may be used in some embodiments to provide a heat map basedon information about where the user's gaze has been focused. A display30 provided on device 10 may allow the user to view this or other visualdata provided by device 10. Merely by way of example, such display 30could include an LCD screen, an LED screen, a prism projector, and/orother display technologies.

In some embodiments of the invention, a gaze-vector can be used tosimulate a perspective from the user's point of view along the gazevector. As described above, the gaze-vector can be used to stabilize animage from scene camera 18 attached to device 10. However, scene camera18 may not capture video of sufficient quality or have the ability totransmit the video in sufficient quality. The gaze vector may thereforebe used along with video data from other cameras, such as stationary ortrack cameras to prepare a composite view that mimics the view fromscene camera 18. Similarly, gaze vectors for one or more users may bedisplayed in an overview perspective. This application of the presentinvention may be particularly relevant to sporting events. For examplein a football game, a gaze vector for an individual player may bedisplayed along with gaze vectors for some or all of the other players.Furthermore, a picture-in picture of the scene-camera view or anapproximated scene-camera view may be displayed for the player that hasthe ball. A similar application may be used in large coordinated policeactions. A person skilled in the art will appreciate that the inventionis not limited to the examples recited above, and that applications withdifferent sports and in other situations may be useful.

The gaze vector may further be used to optimize video data from one ormore scene cameras 18 in a number of ways. In one embodiment of theinvention, the gaze vector may be used to determine focus and lightsettings to scene camera 18. For example, scene camera 18 may be focusedon the target of the gaze vector. Furthermore, the lighting levels ofscene camera 18 may be optimized for the target area of the determinedgaze direction. In embodiments of the invention where computer vision isused to detect objects captured by scene camera 18, information aboutthe distance of the objects from the user may be determined and used tofurther improve the exposure or lighting settings of scene camera 18. Insome embodiments, a gaze target area may be determined within the imagedata representing images of at least a portion user's field of view. Insome of these embodiments, the gaze target area may comprise less thanfive percent, less than ten percent, less than 15 percent, less than 20percent, less than 25 percent, less than 30 percent, less than 35percent, less than 40 percent, less than 45 percent, or less than 50percent of the image data in an image from the scene camera. The controlunit 22 may then control the scene camera to adjust at least one offocus or light sensitivity based on, and/or within, the gaze targetarea.

However, in other embodiments, adjustment of scene camera 18 may not benecessary. For example when a user is driving a car in bright daylightscene camera 18 may capture the dashboard of the car and the viewthrough the windshield. The objects seen through the windshield may beilluminated at a much higher level than the dashboard, and are naturallymuch further away. Using standard autofocus and light detection, scenecamera 18 may be adjusted based on the average brightness in thedetected image and focused to the object in the center of the cameraview. However, when the gaze vector is used, the image may be focused onthe dashboard when the user looks at the dashboard, and focused on theroad when the user looks at the road. Similarly, the lighting levels ofthe captured video may be adjusted to be appropriate for the dashboardwhen the user looks at the dashboard, and to be appropriate for the roadwhen the user looks at the road.

In some embodiments, the gaze data may be used to prioritize areas ofthe image for more detail. Based on the video compression used, prioritymay be given to the area of the image targeted by the gaze. In otherembodiments, the video stream may be divided into two or more feeds. Onelow quality feed may include the entire field of view of the scenecamera, whereas a high quality feed may include a small area around thetarget of the gaze vector. In another embodiment a matrix of video feedsmay be used and their bitrate may be dynamically adapted based on thelocation of the gaze vector. A person skilled in the art will appreciatethat the video quality may be adjusted in a number of different ways andthat the invention is not limited to the examples recited above.

In further embodiments, device 10 may include a speaker for emittingsounds to the user. The speaker may be placed on device 10 in proximityto the user's ear.

In embodiments where a processor is used to detect objects in imagescaptured by scene camera 18, the gaze vector can be used to select onlya subset of the image data around the gaze target to process and thusreduce the amount of processing work to improve feedback time, batterytime etc.

Embodiments of the present invention may employ action triggers whichcause an action to be performed by device 10, some subcomponent thereof,or a connected system, such as a computer, tablet, television, and/orgame machine. According to some embodiments, an action trigger may beenacted by device 10 in many possible ways, including:

-   -   Images captured by sensor 78 and processed by device 10 or a        connected system, resulting in a detection of a present eye, an        absent eye, a blink, a fixation, a saccade, a direction of the        user's gaze or the movement of a user's gaze in a predetermined        pattern.    -   Images captured by scene camera 18 and processed by device 10 or        a connected system, resulting in a detection of changes in        light, identified object(s), identified pattern(s) and        identified gesture(s) using computer vision algorithms.    -   A switch or button based trigger. For example, a physical button        on device 10 or another device.    -   An audio based trigger. For example, sounds, words or commands        spoken by a user and detected by microphone 20.    -   An accelerometer- or gyroscope-detected action trigger, such as        a nod or other head movement.    -   A combination of actions described above.

Embodiments of the present invention include calibration techniqueswhereby external objects may be used to facilitate the calibrationprocess of the device 10. Merely by way of example, illuminators 76,image sensors 78, and/or algorithms of control unit 22 may be adjustedduring such calibration. In some embodiments, an external device such asa printed medium, television, or other display may contain a featurereadily identifiable by device 10. In some embodiments, the feature maybe a specific frame, QR code, invisible feature (i.e., infraredfeature), and/or other visible feature. The feature may contain anidentification code which, when recognized by device 10, allows thedevice 10 to connect to the external device by way of a communicationprotocol such as the internet, Bluetooth, wi-fi or any othercommunications protocol. The external device may then enter acalibration mode whereby icons for calibration are displayed on thescreen and calibration instructions shown or emitted through speakers.Specific calibration processes are well known but typically include anordered display of icons on a screen, which a user then gazes at anddevice 10 determines the gaze direction relative to each displayed icon.

By way of specific example, some embodiments may perform a calibrationmethod whereby when device 10 is worn by a user, and a pattern is placedin front of the user such that the pattern is within the field of viewof the scene camera 18. Scene camera 18 records images of the pattern,while image sensors 78 capture images of the user's eye(s). Theprocessing device analyses the pattern and determines known componentsof the pattern. The processing device analyses the gaze direction of theuser utilizing the images captured by the image sensors 78 and matchesthe gaze direction against the known components of the pattern. As thecomponents are known, the processing device may determine the offsetbetween the location of the components and the determined gaze directionof the user. By knowing this offset, the device is calibrated for theparticular user and can consider the offset when determining the user'sgaze direction relative to other objects.

Further, device 10 may utilize in an algorithm or mathematical modelinformation obtained from analyzing the relationship between a user'sgaze direction and known components of the pattern or similar. By way ofexample device 10 may set a value of a parameter used in an algorithm oreye model used to determine a gaze direction, as would be readilyunderstood by a person skilled in the art.

In some embodiments, additional functionality may be provided whendevice 10 is used in the context of a retail or consumer purchasingenvironment, such as a supermarket. The device may preload informationsuch as personal calibration data to match the profile of a user. Oncedevice 10 is worn by a user it may change from a low power mode tonormal power mode and enter operational mode. Device 10 may allow a userto interactively manage their shopping bill by adding and removing itemsfrom a virtual shopping cart. For example, when a user places an iteminto their shopping cart, they can look at a bar code on the item andswipe right to left with their finger across the barcode to indicatethat the item should be added to the list. Device 10 may recognize thegesture and add the item to a virtual list either on device 10 or at aremote location in communication with device 10. This virtual list maybe controlled by the user for example by removing items by swiping leftto right across a barcode, swiping top to bottom to obtain furtherinformation about an item and gazing at a shopping cart and touching apredetermined location to hear feedback about the list or otherinformation. Upon checking out from the store, the virtual list may beretrieved by a cashier or automatic machine and the user may paydirectly for the items in the shopping cart. In a further embodiment,the user may exit the store directly and upon exiting the store, thevalue of the list may be deducted from a monetary source such as acredit card or account.

In another embodiment, device 10 may perform optical characterrecognition (OCR) on image data including text that is read by a user.For example, scene camera 18 may record images of text read by a user asdetermined by the user's gaze direction. Device 10, or a computingdevice in communication with device 10, may perform OCR on the images ofthe text to determine the words comprising the text. OCR techniques arewell understood and known in the art. Once text has been analyzed usingan OCR technique, it may be sent to a text-to-speech functionality thatreads out the text loud, translated to a result presented to the user inreal time, and/or saved as text data or a string that may be readilymanipulated and understood by a computing device. The text may be savedfor retrieval by a user, or utilized by a computing device or service tounderstand habits of the user. For example, the text may indicate aproduct or service of particular desire to the user. In embodiments, thetext data may be utilized to customize advertisements or the like to bedisplayed to the user.

In embodiments of the invention, device 10 may consider knowninformation such as the size of a barcode or standard object. Forexample a product in a store may include a number of barcodes, onelarger barcode containing the product identifier or SKU, and one smallerbarcode containing the serial number of the particular item in that box.Device 10 may be configured to direct image sensors 78 to only read thelarger code when preparing a shopping list. This may be based on thegaze focus distance for better accuracy. Further, many softwaresolutions such as object recognition libraries require video of astandard size. Accordingly, device 10 may capture images or video onlyof the required size, or crop captured images or video such that theyare in the required size.

FIG. 5 shows a block diagram of one possible method 500 of the inventionfor determining a gaze direction of a user using a wearable frame asdescribed herein. At block 510, as discussed above, device 10 may becalibrated prior to initial use. At block 520, non-image informationsuch as that from movement sensor or positioning device 25 may beacquired by control unit 22. Other types of non-image information mayalso be received as described herein (e.g., information from microphone20). At block 530, image information from scene camera 18 may bereceived by control unit 22.

At block 540, control unit 22 may activate illuminators 76 according tosettings established during calibration at step 510. At block 550, imagedata may be received by control unit 22 from image sensors 78. At block560, control unit 22 may determine if from the information it hasreceived if a gaze direction can be determined as described above. Ifnot, at block 570, illuminators 76, image sensors 78, and/or othercomponents of device 10 may be adjusted as described herein, and thenmethod 500 returns to block 520. In some embodiments, method 500 mayreturn to another step such as block 530 or block 540 if the gazedirection cannot be determined. If control unit 22 can determine gazedirection from the information it has received, at block 580 the gazedirection is determined. Method 500 may repeat at regular or irregularintervals to re-determine gaze direction as needed.

FIG. 6 is a block diagram illustrating an exemplary computer system 600in which embodiments of the present invention may be implemented. Thisexample illustrates a computer system 600 such as may be used, in whole,in part, or with various modifications, to provide and/or control thefunctions of control unit 22, illuminators 76, image sensors 78 and/orother components of the invention such as those discussed above. Forexample, various functions of control unit 22 may be controlled by thecomputer system 600, including, merely by way of example, controllingilluminators 76, receiving images from image sensors 78, processing datafrom image sensors 78, etc.

The computer system 600 is shown comprising hardware elements that maybe electrically coupled via a bus 690. The hardware elements may includeone or more central processing units 610, one or more input devices 620(e.g., a mouse, a keyboard, etc.), and one or more output devices 630(e.g., a display device, a printer, etc.). The computer system 600 mayalso include one or more storage device 640. By way of example, storagedevice(s) 640 may be disk drives, optical storage devices, solid-statestorage device such as a random access memory (“RAM”) and/or a read-onlymemory (“ROM”), which can be programmable, flash-updateable and/or thelike.

The computer system 600 may additionally include a computer-readablestorage media reader 650, a communications system 660 (e.g., a modem, anetwork card (wireless or wired), an infra-red communication device,Bluetooth™ device, cellular communication device, etc.), and workingmemory 680, which may include RAM and ROM devices as described above. Insome embodiments, the computer system 600 may also include a processingacceleration unit 670, which can include a digital signal processor, aspecial-purpose processor and/or the like.

The computer-readable storage media reader 650 can further be connectedto a computer-readable storage medium, together (and, optionally, incombination with storage device(s) 640) comprehensively representingremote, local, fixed, and/or removable storage devices plus storagemedia for temporarily and/or more permanently containingcomputer-readable information. The communications system 660 may permitdata to be exchanged with a network, system, computer and/or othercomponent described above.

The computer system 600 may also include software elements, shown asbeing currently located within a working memory 680, including anoperating system 684 and/or other code 688. It should be appreciatedthat alternate embodiments of a computer system 600 may have numerousvariations from that described above. For example, customized hardwaremight also be used and/or particular elements might be implemented inhardware, software (including portable software, such as applets), orboth. Furthermore, connection to other computing devices such as networkinput/output and data acquisition devices may also occur.

Software of computer system 600 may include code 688 for implementingany or all of the function of the various elements of the architectureas described herein. For example, software, stored on and/or executed bya computer system such as system 600, can provide the functions ofcontrol unit 22, illuminators 76, image sensors 78, and/or othercomponents of the invention such as those discussed above. Methodsimplementable by software on some of these components have beendiscussed above.

The invention has now been described in detail for the purposes ofclarity and understanding. However, it will be appreciated that certainchanges and modifications may be practiced within the scope of theappended claims.

What is claimed is:
 1. A portable eye tracker device comprising: a frameadapted for wearing by a user; at least one optics holding member,wherein the optics holding member comprises: at least one illuminatorconfigured to selectively illuminate at least a portion of at least oneeye of the user; and at least one image sensor configured to captureimage data representing images of at least a portion of at least one eyeof the user; a movement sensor configured to detect movement of theframe; and a control unit configured to: control the at least oneilluminator for the selective illumination of at least a portion of atleast one eye of the user; receive the image data from the imagesensors; and receive information from the movement sensor.
 2. Theportable eye tracker device of claim 1, wherein the control unit isfurther configured to: determine a gaze target area for the user basedat least in part on the image data.
 3. The portable eye tracker deviceof claim 1, wherein the movement sensor comprises: a gyroscope.
 4. Theportable eye tracker device of claim 1, wherein the control unit isfurther configured to: determine a gaze target area for the user basedat least in part on the image data and information received from themovement sensor.
 5. The portable eye tracker device of claim 1, furthercomprising: a scene camera facing away from the user and configured tocapture image data representing images of at least a portion of a viewof the user.
 6. The portable eye tracker device of claim 5, wherein thecontrol unit is further configured to: determine a gaze target areawithin the image data representing images of at least a portion user'sfield of view wherein the gaze target area comprises less than tenpercent of the image data in an image from the scene camera; and controlthe scene camera to adjust at least one of focus or light sensitivitybased on the gaze target area.
 7. The portable eye tracker device ofclaim 6, wherein: all illuminators emit the same wavelength of light. 8.The portable eye tracker device of claim 1, wherein the control unit isfurther configured to: control a first subset of the at least oneilluminator to selectively illuminate at least a portion of at least oneeye of the user; receive a first set of image data from at least oneimage sensor of the first subset, wherein the first set of image datarepresents images of at least a portion of at least one eye of the usercaptured when at least one eye of the user is exposed to light from thefirst subset of the at least one illuminator; determine that a gazetarget area cannot be determined based on the first set of image data;control a second subset of the at least one illuminator to selectivelyilluminate at least a portion of at least one eye of the user; receive asecond set of image data from at least one image sensor of the secondsubset, wherein the second set of image data represents images of atleast a portion of at least one eye of the user captured when at leastone eye of the user is exposed to light from the second subset of the atleast one illuminator; and determine the gaze target area for the userbased at least in part on the second set of image data.
 9. The portableeye tracker device of claim 1, wherein: the portable eye tracker devicefurther comprises a positioning device configured to determine aposition of the portable eye tracker device; and the control unit isfurther configured to determine a gaze target area for the user based atleast in part on the image data and information received from thepositioning device.
 10. The portable eye tracker device of claim 1,wherein the control unit is further configured to: receive image datafrom at least one image sensor; determine a level of ambient light fromthe image data; adjust a brightness or exposure time of at least oneilluminator based at least in part on the level of ambient light. 11.The portable eye tracker device of claim 1, further comprising: adisplay viewable by the user.
 12. The portable eye tracker device ofclaim 1, wherein the control unit is further configured to: receiveimage data from at least one image sensor representing images includingareas besides the user's eyes.
 13. A method of determining a gazedirection for a user, comprising: activating at least one illuminator ona frame worn by a user to selectively illuminate at least a portion ofat least one of the user's eyes; receiving image data representingimages at least a portion of at least one of the user's eyes from atleast one image sensor on the frame; receiving information from amovement sensor configured to detect movement of the frame; anddetermining a gaze target area for the user based at least in part onthe image data and information from the movement sensor.
 14. The methodof claim 13, wherein the movement sensor comprises: a gyroscope.
 15. Themethod of claim 13, further comprising: receiving images captured by ascene camera representing images of at least a portion of a view of theuser.
 16. The method of claim 15, further comprising: determining a gazetarget area within the image data representing images of at least aportion user's field of view wherein the gaze target area comprises lessthan ten percent of the image data in an image from the scene camera;and controlling the scene camera to adjust at least one of focus orlight sensitivity based on the gaze target area.
 17. A non-transitorymachine readable medium having instructions thereon for determining agaze direction for a user, the instructions executable by a processorfor at least: activating at least one illuminator on a frame worn by auser to selectively illuminate at least a portion of at least one of theuser's eyes; receiving image data representing images at least a portionof at least one of the user's eyes from at least one image sensor on theframe; receiving information from a movement sensor configured to detectmovement of the frame; and determining a gaze target area for the userbased at least in part on the image data and information from themovement sensor.
 18. The non-transitory machine-readable medium of claim17, wherein the movement sensor comprises: a gyroscope.
 19. Thenon-transitory machine-readable medium of claim 17, wherein theinstructions are further executable for at least: receiving imagescaptured by a scene camera representing images of at least a portion ofa view of the user.
 20. The non-transitory machine-readable medium ofclaim 19, wherein the instructions are further executable for at least:determining a gaze target area within the image data representing imagesof at least a portion user's field of view wherein the gaze target areacomprises less than ten percent of the image data in an image from thescene camera; and controlling the scene camera to adjust at least one offocus or light sensitivity based on the gaze target area.